Skip to content

blinkd: Do not wait for service startup during boot #64

Merged
merged 1 commit into from
Jan 4, 2019

Conversation

donald
Copy link
Collaborator

@donald donald commented Jan 4, 2019

Fixes #54

When systemd-udev-trigger.service is started, it uses /bin/udevadm settle to trigger udevd to process the rules for discovered devices and
waits until this is done.

The rules file 51-blink.rules asks udevd to run systemctl start blinkd. By default, systemctl start waits until the target service
is started.

blinkd.service has a After= dependency on sysinit.target (via
DefaultDependencies) and waits for that.

sysinit.target has a After= dependency on systemd-udev-trigger.service
and waits for that.

So we have a deadlock.

Prevent deadlock by adding --no-block to systemctl start.

Note 1: This fix will prevent the deadlock. However, blinkd will
not start on boot currently, because it depends on python3 from
a /pkg path and this is not available yet.

Note 2: Same fix should be applied to /etc/udev/rules.d/51-barcodereader.rules

Fixes #54

When systemd-udev-trigger.service is started, it uses `/bin/udevadm
settle` to trigger udevd to process the rules for discovered devices and
waits until this is done.

The rules file `51-blink.rules` asks udevd to run `systemctl start
blinkd`. By default, `systemctl start` waits until the target service
is started.

`blinkd.service` has a `After=` dependency on `sysinit.target` (via
`DefaultDependencies`) and waits for that.

`sysinit.target` has a `After=` dependency on `systemd-udev-trigger.service`
and waits for that.

So we have a deadlock.

Prevent deadlock by adding `--no-block` to `systemctl start`.

Note 1: This fix will prevent the deadlock. However, blinkd will
not start on boot currently, because it depends on python3 from
a /pkg path and this is not available yet.

Note 2: Same fix should be applied to `/etc/udev/rules.d/51-barcodereader.rules`
@pmenzel pmenzel merged commit e526cf4 into master Jan 4, 2019
donald added a commit that referenced this pull request Jan 7, 2019
This is the second part of the blinkd fix (issue #54).

PR #64 prevented the deadlock during service start.

However, currently blinkd is not starting at system boot, because
it uses `env python3` in the shebang. This will activate our default
(big, scientific) python3 which is located in /pkg and depends
on the automounter.

One solution to this problem would be to add After=automount.service
to blinkd.service. This would delay the startup of blinkd to the
point, when automounter has finished startup. However, experiments
showed, that this doesn't work. It looks like the startup
command of automount.service exists after daemon process has been
forked but before the automount paths have been mounted. So
systemd regards automount.service has started before the automount
paths are really available and services with After=automount.service
are still started to early to use them.

That aside, although /usr/bin/python3 depends on /pkg, it is not
supposed to go via NFS under normal circumstances, because of the
local mechanism on mxpkg. However, under exceptional circumstances
(e.g. transition to a new python3 version) it might go over
NFS and we don't want long-running deamons to lock nfs mounts.

A python3 with the blink1 package from PyPi has been added
to the system [1] and is available as
/usr/local/system/python3/bin/python3.

This python installation is guaranteed to be on a local
disk, so use this path in the shebang.

[1] mariux64/pkg-scripts@8e149c1
donald added a commit that referenced this pull request Jan 7, 2019
This is the second part of the blinkd fix (issue #54).

PR #64 prevented the deadlock during service start.

However, currently blinkd is not starting at system boot, because
it uses `env python3` in the shebang. This will activate our default
(big, scientific) python3 which is located in /pkg and depends
on the automounter.

One solution to this problem would be to add After=automount.service
to blinkd.service. This would delay the startup of blinkd to the
point, when automounter has finished startup. However, experiments
showed, that this doesn't work. It looks like the startup
command of automount.service exists after daemon process has been
forked but before the automount paths have been mounted. So
systemd regards automount.service has started before the automount
paths are really available and services with After=automount.service
are still started to early to use them.

That aside, although /usr/bin/python3 is not to go via NFS under
normal circumstances because of the loacl mechanis of mxpkg, under
expectional circumstances this can happen (e.g. when the python3
version is updates). We don't want long-running deamons to lock
nfs mounts, so this would not be a good solution, even if made to
work.

A python3 with the blink1 package from PyPi has been added
to the system [1] and is available as
/usr/local/system/python3/bin/python3.

This python3 installation is guaranteed to be on a local
disk, so use this path in the shebang.

[1] mariux64/pkg-scripts@8e149c1
donald added a commit that referenced this pull request Jan 7, 2019
This is the second part of the blinkd fix (issue #54).

PR #64 prevented the deadlock during service start.

However, currently blinkd is not starting at system boot, because
it uses `env python3` in the shebang. This will activate our default
(big, scientific) python3 which is located in /pkg and depends
on the automounter.

One solution to this problem would be to add After=automount.service
to blinkd.service. This would delay the startup of blinkd to the
point, when automounter has finished startup. However, experiments
showed, that this doesn't work. It looks like the startup
command of automount.service exists after daemon process has been
forked but before the automount paths have been mounted. So
systemd regards automount.service has started before the automount
paths are really available and services with After=automount.service
are still started to early to use them.

That aside, although /usr/bin/python3 does not to go via NFS under
normal circumstances because of the loacl mechanis of mxpkg, under
expectional circumstances this can happen (e.g. when the python3
version is updates). We don't want long-running deamons to lock
nfs mounts, so this would not be a good solution, even if made to
work.

A python3 with the blink1 package from PyPi has been added
to the system [1] and is available as
/usr/local/system/python3/bin/python3.

This python3 installation is guaranteed to be on a local
disk, so use this path in the shebang.

[1] mariux64/pkg-scripts@8e149c1
donald added a commit that referenced this pull request Jan 7, 2019
This is the second part of the blinkd fix (issue #54).

PR #64 prevented the deadlock during service start.

However, currently blinkd is not starting at system boot, because
it uses `env python3` in the shebang. This will activate our default
(big, scientific) python3 which is located in /pkg and depends
on the automounter.

One solution to this problem would be to add After=automount.service
to blinkd.service. This would delay the startup of blinkd to the
point, when automounter has finished startup. However, experiments
showed, that this doesn't work. It looks like the startup
command of automount.service exists after daemon process has been
forked but before the automount paths have been mounted. So
systemd regards automount.service has started before the automount
paths are really available and services with After=automount.service
are still started to early to use them.

That aside, although /usr/bin/python3 does not to go via NFS under
normal circumstances because of the local mechanism of mxpkg, under
expectional circumstances this can happen (e.g. when the python3
version is updates). We don't want long-running deamons to lock
nfs mounts, so this would not be a good solution, even if made to
work.

A python3 with the blink1 package from PyPi has been added
to the system [1] and is available as
/usr/local/system/python3/bin/python3.

This python3 installation is guaranteed to be on a local
disk, so use this path in the shebang.

[1] mariux64/pkg-scripts@8e149c1
donald added a commit that referenced this pull request Jan 7, 2019
This is the second part of the blinkd fix (issue #54).

PR #64 prevented the deadlock during service start.

However, currently blinkd is not starting at system boot, because
it uses `env python3` in the shebang. This will activate our default
(big, scientific) python3 which is located in /pkg and depends
on the automounter.

One solution to this problem would be to add After=automount.service
to blinkd.service. This would delay the startup of blinkd to the
point, when automounter has finished startup. However, experiments
showed, that this doesn't work. It looks like the startup
command of automount.service exists after daemon process has been
forked but before the automount paths have been mounted. So
systemd regards automount.service has started before the automount
paths are really available and services with After=automount.service
are still started to early to use them.

That aside, although /usr/bin/python3 does not to go via NFS under
normal circumstances because of the local mechanism of mxpkg, under
expectional circumstances, this can happen (e.g. when the python3
version is updates). We don't want long-running daemons to lock
nfs mounts, so this would not be a good solution, even if made to
work.

A python3 with the blink1 package from PyPi has been added
to the system [1] and is available as
/usr/local/system/python3/bin/python3.

This python3 installation is guaranteed to be on a local
disk, so use this path in the shebang.

[1] mariux64/pkg-scripts@8e149c1
donald added a commit that referenced this pull request Jan 7, 2019
This is the second part of the blinkd fix (issue #54).

PR #64 prevented the deadlock during service start.

However, currently blinkd is not starting at system boot, because
it uses `env python3` in the shebang. This will activate our default
(big, scientific) python3 which is located in /pkg and depends
on the automounter.

One solution to this problem would be to add After=automount.service
to blinkd.service. This would delay the startup of blinkd to the
point, when automounter has finished startup. However, experiments
showed, that this doesn't work. It looks like the startup
command of automount.service exists after daemon process has been
forked but before the automount paths have been mounted. So
systemd regards automount.service has started before the automount
paths are really available and services with After=automount.service
are still started to early to use them.

That aside, although /usr/bin/python3 does not to go via NFS under
normal circumstances because of the local mechanism of mxpkg, under
exceptional circumstances, this can happen (e.g. when the python3
version is updates). We don't want long-running daemons to lock
nfs mounts, so this would not be a good solution, even if made to
work.

A python3 with the blink1 package from PyPi has been added
to the system [1] and is available as
/usr/local/system/python3/bin/python3.

This python3 installation is guaranteed to be on a local
disk, so use this path in the shebang.

[1] mariux64/pkg-scripts@8e149c1
@donald donald mentioned this pull request Jan 7, 2019
Sign in to join this conversation on GitHub.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

udev blink rule delays boot by 3 minutes
2 participants